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I introduction 


When we look at the silhouettes in Piwsn's work uf Spring" (figure >). w * 

perceive them in terms of very particular shapes, some familiar, some less so. This is 
quite remarkable, because the silhouettes Quid In theory have teen generated by an inFmue 
variety of shapes which, from other viewpoints, have no discernible similarities to :he 
Shapes we perceive. One can perhaps attribute part of the phenomenon to a familiarity 
With the depicted shapes; but not all of it, because one can use- the medium, of a silhouette 
to convey a new Shape, and because even with considerable effort it is difficult to imagine 
the more biiarr* three-dimensional surfaces that could have given rise to the same 

silhouettes. , 

This phenomenon is or quite general importance For the analysis of an 

The boundary of a silhouette is simply one type of acduding contour {see c^= Waltz 1975), 
and such contours are an artist's principal msatu of ccmveyLng information about shape. 
The paradox is that they apparently tell us more about shape than they should. For 
example, neighbouring points on such a contour could an general arise from widely 
separated points on the original surface, but our perceptual interpretation usually ignores 

this pOfcsLbiUty. 

ThlS means that Implicit in the way wfr interpret ati occluding contour, there must 
he some a pTitrrt assumptions that allow US to infer a shape from an outline. If a surface 
violates these assumptions, our analysis wilt be wrong, in the sense that the shape we assign 
TO the contour* will differ from the shape that actually caused them. An everyday example 
erf this phenomenon IS the shadowgraph, where the appropriate arrangement of one s hands 
an, to the surprise and delight of a child, produce the shadow of an apparently quite 
different shape, like a duck or 1 rabbiL 

What assumptions is it reasonable » suppose that we make? 1 shall argue for 
these two;, {a) that nearby points Ofi a contour correspond TO nearby p&JntS on the viewed 
surface; and (b) that the distinction between convexities and concavities in a contour 
reflects real properties of the surFace, not an artifact of perspective. 

Some iurfaceS seen from some Viewpoints will satisfy these conditions, and some 
will not. Our first task ii to understand what it is about a surface that makes it satuFy these 
assumptions, and the mam remit of the first part of the paper achieves this. Theorem 1 
shows that, if the assumptions (a) and (b) hold for all distant vantage points such that the 
line Of Sight lie* parallel to some fixed plane, then the viewed surface must he a geiwrattztd 
cone, (A generalized cone is the surface swept out by moving a cross-section of fixed shape 
but smoothly varying site, along an axis, as illustrated in figure 5) 

This result is strong and surprising. It means that if one has a method for 
interpreting contours that relies on assumptions {a) and (b), then the method implicitly 
assumes that the viewed shape is a generalized con*. One can think of such a method ai 
first throwing a generalized cone blanket round the viewed shape, and Then describing *.he 
shape Of the blanket. This in turn means that the representation of S-D shape that IS 
subsequently used can. without further lot! trf information, be based on generalized cones 
{like that of Marr Sc Nishihara 1977)- 



1- "Rites of spring" by Picasso, We immediately interpret the silhouettes in terms of 
particular 3-D surfaces, despiLe the paucity of information in the image, In order to do this, 
we must be Sorssig’j 11 g additional assumptions and constraints to bear on the analysis of these 
contours* shapes. This article enquires about the nature or this a priori information. 


The remit ti of practical importance, became st can be used as a basil for methods 
that interpret rccludinE; contours. Thu is the point of the later sections of the article, where 
we Shall assume that the viewed, object Js a gencraliied cane, and ask how to discover its 
specification* (its axis and cross-section) using only the contours that are visible in IW (mage. 
The second secuon deals with the image of a single generaliied cone, and the third with 
objects that arc Coftipdsed of several tunes joined together in various ways. These methods 
will too successful provided that hO axis appears severely foreshortened in the image. Vatan 
^916) and Vatah & Mar: (1977) exhibit algorithms based on the theory, and the results of 
applying them to a number of natural images, 

The main body Of this article attempts to set tnl! the motives and results of this 
research in plain English. I hope that it will be accessible to the general reader. The 
appendix contains precise statements uf the restrictions and theorems, and gives their proofs. 
Most Of the arguments there are geometrical and a specialized mathematical background is 
not necessary to understand them. 

/dotation 

As far ®t possible, 1 shall adhere to the following convention*. Surfaces, curves 
and lines in three-dimensional space will he denoted by upper-case Greek letters (E, Dt 
curves and Lines in an image from viewpoint V will be denoted by suffixed upper-case 
Roman letters (5^, and upper-case Roman letters without a suffix (F, Q) denote 
points, cither in 3-ipace or in the image: Lower-case letters obey the usual convention*, so 
that f r g, * and p are function*, a, ff, $ and 4> arc angles and *, y, r are coordinates, 

ll The basic hypotH^ses and their irnplications 

Our discussion will centre on the four structures that appear in figute % The« arE 
(3) some 3-D Surface Z\ C2) «S image or jiihouettE Sy as seen from a viewpoint V\ {3} the 
bounding contour Cy of Sy; and (4) the set Of points on th« surface E, that project onto 
the- contour Cy. We shall call this last jet che contour genmiMr of Cy, and we shall denote it 

by iy 

We express the assumptions, to which t referred in the introduction, as restrictions 
on portions of the surface E or its image. 

Jter:ric£j«t Rl:Tht JUr/cce E ti jmflcJfe, 


We make this restriction only because we need to be able to* distinguish convex pieces of 
contour from concave ones, and. it is easiest to do this if the surfaces and contours in 
question can be differentiated twice, (In fact, we could have allowed the contours in the 
image to be composed oF straight line segments. Since the notions of 'Convex and concave 
have a well-defined meaning m such cas&j; but tittle is gamed by doing this). 


Rtttrtitlcn ft^rfiacin pertu: on canfour generator iy projttts m s dijftrmt pofnf on 



2. The three-dimensional surface 2, viewed from a point V, forms the silhouette Sy in the 
image vie the imaging process s. The boundary of Sy, obtained by the boundary operator d 
is denoted by Cy and we call it the contour of 2. The set of points on 2 that i maps onto 
Cy we call the contour generator of Cy, and it is denoted by Ty. The map from 2 to Ty 
induced by d is denoted by S 




















the COnlffiLT Cy. 


This means that each line of sight from V tc the edge of I - that Is, from V to Ty ~ 
couches £ at -anly one point, not at two points (as shown in figure JO) or along a line 
segment- The condition that each line of sigh: touches Z at one, rather than at two or a 
finite number of points, is equivalent to saying that Z is convex, as seen: from this viewpoint 
{see figure ?03. This is not as strong a condition as it appears at first sight, because in 
practise IE will not usually be imposed for ail Viewpoints (e.g. theorem I}, and there are ways 
of regaining the generality that it excludes (theorem 5). Forbidding the line of sight from 
touching £ along a line segment {as can happen for example if one views a cube from a 
direction parallel with one of lu faces) is only a technical restriction;; one can escape it 
Without changing the Situation in in important way by deforming the Viewed surface very 
slightly. 

flej-rricridFi R3: Nt&rbf point; on th( emtwr Cy arise Jr am nearby points flfl lAt centaur 

gnurater Ty- 

This condition is a powerful one, and is best explained by figure 3. Suppose that the 
contour a& of figure 3 really arose from two hitls, but the dotted portion of i happened to be 
Invisible. Then the contour generator of ai> would be discontinuous at P, where it leaps 
from one hill to the next. This u the situation that jR? forbids, and it is essentially 
equivalent ro assuming that the image contains no invisible obscuring edges. Together, R2 
and RS imply that the contour generator Tp is a continuous curve across £ - i.e, that it 
does not jump erratically from plaM to place on £. R3 ii a strung condition, but without it 
□ne can say almost nothing about £ and I know of no way to proceed without it. 
Fortunately it is obeyed by most real-world images. 

jRemcrJt 

In fact, R2 and R3 are not quite independent, since if one assumes that the 
surface £ is bounded, ^i.? is a consequence of R2. T& see this, notice that at points like P In 
figure 3 where R? IS violated, the viewing ray to P graces both hllh, and so causes a 
violation of R2. Nevertheless, the two restrictions have sufficiently different meanings to 
make it worth stating them separately. 

Wring pnirtti cf inflttiOl 3 

The restrictions Rl-R} are very general, and guarantee only the integrity of Cy 
and Ty. not their interpretahility. L« us therefore suppose that a contour Cy, like chat 
shown tn figure d, was obtained under conditions that satisfy these restrictions, and enquire 
what properties of Cy we can rely upon. Clearly, no nwital properties of Cy ear be used, 
because Cy arises from viewing a surface £ at an unknown orientation — i.e. through at 
best a linear operator, and such operators do not preserve distances. The values of C^’s 
max ima and minima, and their separation, remain uninformative unfit substantially more is 
known about £ and ihe perspective from which it is being viewed. But the qualitative 
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4. A typical piece of contour. Unci] we have some information about the nature and 
position of the surface that gave rise to it, the only features that we can use are its points of 
inflexion. 


nations of maxima and mLnLma on a planar curve an presided by a linear operator — that 
ii, the distinction between convex and concave is Invariant. This fact is captured by Lemma 
l of the appendix. 

Let US therefore suppose that we have been presented with a contour segment lit* 
that shown in figure 1 Restriction. R} guarantees that adjacent points on the contour arise 
from adjacent points on the surface %, but no metrical features are yet reliable. The only 
straightforward feature that remains is the distinction between a convex contour segment 
and a concave one, which rests in turn on the notion of an JnfleXLMi point. For a general 
surface E and contour generator Tj/. even paints of inflexion in Cy wilt often be 
meaningless, and to attribute significance to them is to make an additional assumption about 
E- So we next ask, haw exactly should we formulate the assumption that points of inflexion 
are significant? 

The restrictions RI-R3 allow us to thmk oF Ty, the contour generator of Cy on E, 
as a smooth piece nf Wire bent in 3-Spn.Ce. For inflexion points on Cy to be significant 
however, lemma l (see the appendix) tells us thai wr need two things; ft} the transformation 
due to the imaging process that produces Cy mus; be linear, and {?) the curve on which that 
transform acti nniiiT lie tn a plane. Because the general perspective transformation is not 
linear, condition ([} tells us that our whale theory applies only to distant viewing points, 
became only in these conditions is the imaging process a linear projection. Condition (2) 
informs us that the convexsoncave distinction on be meaningful in Cy only if the bent 
wire that is Ty lies in a plane. This gives ms our fourth condition. 

jRejfrtc-riflit R4: Thi tonlma generator Ty of Cy ii planar. 

This condition IS a Strong one, and Sharply delimits the class of admissible surfaces X- There 
seems however ta be na way of avoiding it if one wishes to use the distinction between 
convex and, concave contour segments. 

Implications of tbe four restrictions: 

A generalized cenr, Illustrated in figure £. is defined to be the surface swept duL by 
moving a simple smooth cross-section along some axis, at the same time magnifying or 
contracting it in a smoothly varying way. This etoss-jeetton is defined by the function 
pir t 9) * 0, and when the cross-section is convex, we shall use cylindrical coordinates r - 
The magnification of the cross-section at each pain: Ls specified by the function h(zh 
where x is the distance measured along the nme's axis. The axis itself will be labelled. A- 
Notice that in general the t axis need not be perpendicular 5a the plane % - Q of the cross' 
section- These conventions are illustrated by figure -5. 

We may demand that the restrictions R2 - R4 hold for all views, or for a subclass 
of the possible views of E. If we demand that they hold for only one (distant) viewpoint, 
this imposes no interesting restrictions on the nature of E. Theorem l studies the two 
dimensional ease, when the restrictions a?e assumed ;o bold for all distant viewpoints whose 



5, The definition of a generalized cone, In this article, a gencr^Licd, cone is the surface 
generates by moving u. smooth cross-MctLon fl along a. straight axis The cross-section may 
vary smoothly in size (as prescribed by tbe axial scaling function but its shape remains 
constant. The eccentricity of the cone is the angle ^ between its axis an.cl a piano tontaibrng 
a cross-section. 













lines of sight lie parallel 10 some fixed plant, and it is the most interesting result of the 
section. Finally, theorem 2 studies the consequences of assuming thu the restrictions hold 
for all distant viewpoints. 

The tcuic thme ru 

Cartier, we d-e-Tmed Jy to be the image of 2 as seen from the vantage point V 
(figure 2). ThLs is equivalent to saving that Sy is the perspective projection of 2 from the 
point In- theorem ]„ We shall make two simplifying assumptions about the projection Sy„ 
first that the projection IS orthogonal, Which is approximated when the vantage paint V is 
very distant from £ compared to its site, and second, that the viewing directions are 
eon fined to a plane H round £ and which intersects E. We deal in some seme only with 
'side" views of 2, ana are forbidding "end-om” views. Such projections arc completely 
specified by the direction of the vantage point from 2 In the confining plane II, and we 
denote this by the angle A Wr shall use the notation 3^, and in place of Sy, Cy 
and IV to indicate that the above restrictions are Ln effect. The proofs of theorems 1 and 2 
are set cult in the appendix. I give here their statements in plain English 

Theorem I. I is a generated cone with convex cross-section if and only If 
Rl is satisfied, and R2 - R4 are satisfied for all orthogonal projections $ 
associated with some plane IL in the sense defined above. This plane Lies 
parallel to the cross-secCion of the cone. 

Theorem 2, 2 satisfies Rl and R2 - R.4 for alt dislant vantage points V if 
and only if 2 is a quadratic surface. 


flfwisrAs aboa! theorems t and 2 

It is theorem 1 that allows the crucial stop for the overall argument. It says that if, 
for distant viewpoints whose viewing directions lie parallel io some plane, a surface’s shapE 
can successfully be inferred using only the convexities and concavities of its bounding 
contours in an image, then that surface is a generaliied cone with convex cross-section, or is 
composed Of them. Hence if one that one can discover a Surface's shape from such 

information, then this is equivalent to assuming that the viewed surface is a generaliied 
«me. The assumption of theorem 3, about orthogonal projections parallel co the plane of 
the cross-section, is tolerable because as we shall see the methods to which the theory fives 
rise usually degrade only slowly as hie ttioves nearer, increasing the ejects of perspective, or 
out of [He plane of the cross-section. Furthermore there does appear to be something special 
about the perception of view? that look down the s-axis of the figure (see the remarks madE 
by Marr Sc Nishihara (19T?) about Warrington Sc Taylor’s (1973) "unconventional views"), 

Theorem 2 is in wresting, because it shows how very strong our restrictions are, 
One can gain a fesl for how the planar condition R4 fails for higher-order surfaces by 
studying the behaviour of y2n 4 « / ( SK -f,g Ure &). T’tiia Surface is a sphere for n 

- /, and tends to a Cube as n grows large, The contour generator fp/, which is a cirdE for u 


" I (figure 6a}, becomes the outline marked “with thick fines in figure 5b for high values of 
n. This; contour-generator ii clearly not planar; as n increases, the lower third of the contour 
generator is pulled to wards the viewer, and the upper third is pushed away. 

These results provide a further argument for using something based on 
generalised cylinders (Bmford 1971) for the internal representation of shape (see Marr & 
Nishihara I&77), an argument based nut on- utility, at mast other justifications are, but tin the 
assumption* implicit in the decoding Of an image. It is indeed extremely fortunate that 
many important three-dimensional structures tan be closely approximated by a few 
generalised cones, although i: is not accidental that objects whose shape was achieved by 
growth tike limhs and Stalagmites, can be so approximated. 


2: Interpreting the image of a. single genarnlized cone 

Theorem 1 essentially tells us that, when trying to infer the shape of a surface 
from its bounding contours, in an image we cannot avoid assuming that the surface is a 
generaliled cone. We are now faced with an obvious question. If we assume that out data 
consists of contours in the image of a gen a allied cone, how may w? interpret them? To 
specify a gCMMliied, cone, we have to Specify its axis A, CfOSi-SKtion p($). and axial scaling 
function how can we discover them from an Imager" 

The answer to this question commence* with theorem 3. which shows how the 
occluding contours in an image may be used ro find the "image" of the cone's axis for those 
distant viewpoints that lie in the privileged viewing plane referred to in theorem I. Jn 
general, of course, our viewpoint will not tie in this plane, and so we have fo examine the 
stability of this result as the viewpoint moves out of the plane. Thu is achieved by theorem 
4. which introduces a new concept called the jArieron of a generalised cone. ThE skeleton is 
Hot a difficult idea, however, unce it is very like the sec of lihts a cartoonist draws to convey 
the shape of a curved object, The idea of a skeleton allows us to extend the theory to 
generalned cones whose cross-section is not convex. Requiring the restriction R2 to hold for 
all ^-projections essentially forbids tins clan of cones, and I said earlier that one can 
circumvent this restriction in practise. Theorem 5 shows how. Finally, there is a short 
discussion about cases in which the cone ts viewed from a nearby rather than from a distant 
point, and cases in which the axis of the cone is not a Jtratght Line. 

The Overall purpose of she section is io give a set of methods for interpreting the 
image of a Single generaliied COhc. The methods derived here will not succeed for all views; 
they will fail when the image of the cone's axis is substantially foreshortened, It is part of 
the overall theory that such views have to be handled differently (Marr k Nishihara 1977}, 

Ftndlng (Ar axil from a favaumble view 

Provided that the viewed surface is a generalised tone, and that the viewing point 
Satisfies the conditions of theorem 1, the axis of the cone may easily be determined by the 
rough symmetry formed around it. 

Theorem. 3 (Axtai Jymtdwryj. Let H be a generaliied cone with convex cross- 


G. The surface * ylv. + s 2n . j f or n m j (figure 6a) and n - 1000 (figure 6b). The 
contour generator is shown Lrt thirl Lines. It is planar (l Circle) Eft 6a, but not for n > 1. 
Figure 6b shows how the contour generator is pulled. oo? of a plane far high values of n. 
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■section p(&), and kr the cross-scttLon sca.ling function AfaJ contain ai least 
one concavity, Th.cn fo: all viewing directions ^ 

(i) the silhouette of £, decomposes into n > 2 contour segnrenEs by 
spirting ic at points of inflexion; 

CiO the image of the axis A of £ establishes an axial symmetry between at 
least fu - 2 ) contour segments, including all concave ones. Corresponding 
segments are either both convex or both concave; 

(iii) the ratios of the distances of corresponding segments either side of the 
axis of symmetry are all the same, 

Corolfary- The image of the axis of £ is uniquely determined if there exists only one such 
axial symmetry. 

This theorem is best explained by tooting at figure 7. Here* we see that the 
contour divides at inflexion points inso three segments, labelled Cj to Cy, The two concave 
Moments Cj and Cj are roughly symmetric about the image of the cone's axis A, although 
their distances away from the axis may not be equal The third clause of the theorem states 
that, jf Cj is half as far from A as Cjj, then the same will be true of all other segments that 
correspond under the symmetry. We shall call the type of symmetry established by theorem 
3 a guflitfetiv* lyjTtwJry. Its important features are {a) that It holds between convex or 
concave of a contour, and (b) that it includes a scaling factor, 

Although there Is a pOitir-wise symmetry between the contour on the two sides of 
tho axis, unless the viewed surface is a right generalised cone and Che contours are 
faithfully diagnosed in the image, such symmetries are expensive to detect- A qualitative 
symmetry, on Che other hand, does riot have to be found on a pomt-by-point basis. This is 
important because it miles fin ding the symmetry, and hence its axis, a practical 
computational proposition. By dividing the contour into convex and concave segments and 
noticing that Ihc symmetry preserves this distinction, we have greatly reduced the number of 
items that have to be examined and made the computational load, acceptable. 

There is one other point of importance about this result and it comes from the 
corollary, which says that if only one symmetry exists among the contours, the axis of £ is 
determined uniquely. This means that the analysis of contour is self ■checking, and one does 
not have to appeal to the "famtltarity* of the deduced shape to know that one has a valid 
interpretation nf the image. This is of course essential if one is to be able to analyte novel 
shapes. The reader will observe that all of the theorems, that are directed at the analysis of 
contour, have uniqueness corollaries like that of theorem 3. It is on these that the algorithms 
themselves will rest most derectly. 

VttWing ffirefrimj mar te^anur wi-fn a ron/s croji-iftUnn 
We next ask what happens to the generalized cones of theorem 3 if the viewpoint 
remains distant, but if She viewing direction moves out of it! constraining plane It? As long 
as the image Sy approximates an orthogonal projection parallel to the plane of the 
generalised core's tross^ection, variations in the silhouette of I are due to changes in the 
scaling function Ml) along the tone's axis A. as illustrated by figure &a. On the other hand. 
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when [he viewpoint is moved so drastically [hat [he viewing direction li« parallel to A* the 
silhouette of X is due entirely to fid), its cross-section function (figure &b) r and Is in fact due 
to the Cross-section of U at the point where frft) achieves its maximum value, 

At both of These extremes the COntaui generator Ty of S is planar, but for other 
viewing directions it n«d not be. This Js obviously true for a dumbell shape, or for an ice¬ 
cream cone, hut theorem 2 assures is that it is mare insidiously true even for a surface as 
simple as an egg. where the contour generator is a circle for the end-view, a near ovaE for a 
sLtk view, and slides from one to the other in between (sea figure iCb below). Another 
example as Ehe surface shown in figure 6c; the contour generator frotn this view is clearly 
no? planar. 

How are we to handle such views? For convex objects tike an egg, where the only 
occluding contours arise from its silhouette, there is very little more one can do to infer its 
shape when seen from one of the intermediate views, unless one knows something about the 
orientation of the egg relative 10 the viewer. For object* thai are not convex, like the double 
spike of figure S, one can separate the contours that arise in the image into two classes; 
those approximately due to the “sides - of the figure {the two spikes separated in figure 3d), 
and those approximately due to a cross-sfttion, like the central ellipse In figure fid. 

This division gives us our main tool for analysing non-standard views, and it is 
best explained with the help of figure 9. Suppose that a generalized cone X is being viewed 
frotn a distant point V and the line of sight Is not parallel io The plane of the cane’s cross- 
section, The contour generator for viewpoint^ Is approximated by two components- One 
is easy to define, it is the places on E where the size of the grass-section is stationary — that 
is, where AfiJ achieves a maximum or minimum For an egg, it is the fattest p-oss’sacticjn, 
and other examples are shown shaded in figure (0. We call these curves radial 
and denote them by T “■ notice there is no suffix, since T does not depend on the vantage 
point, The other idea we have to make as precise as possible is what we mean by the 'sides - ' 
nf E from a viewpoint such At y, and for this we make the construction illustrated in figure 
Vl/e drop a perpendicular from V to W, which does lie in the standard viewing plane. 
Then the contour generator for viewpoint V IS approximately the projection of the contour 
generator Tyi for yt, which lS simply Fa for some angle flj. For example, in the case of an 
egg of length l and diameter d r the skeleton (shown in figure EGb} has length f-iinfa) and 
width d, when viewed in a plane containing the egg's major axis at an angle x to For 
angles where > d, this is a reasonable approximation, and when < *f, we 

have an “unconventionar view. 

The reason why the skeleton is a useful construct for recognition is that one can 
dececr iis presence in an image by the many relationships that exist among its parts. In face, 
w* can use these relationships to set up constraints on a set of occluding contours such Thai 
if those constraint are all satisfied by a unique interpretation of the contours in the image, 
we can be reasonably certain that we have found a skeleton, and hence can interpret the 
contours as arising from a generalized cone I whose axis is then determined. The relation 
themselves consist of qualitative symmetries and parallelism, and are preserved by an 
orthogonal projection, Hence provided, chat the contours as seen from vantage point V in 
figure 9 are approximately the projection of th-e contours as seen from V { , the relations wjtf 
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10. Example* of skeleton* of surfaces, a* ■defined in the Test and m figures £ and a. The 
cruss-.sect.iQ ns responsible for [he rad III -ememirks of the surfaces are shown shaded. ThE 
skeleton ceases to be a reasonable approximation w the contours [fiat occur m the image 
whenever the viewing angle is such as to make the projection of the length of :he cone less 
than the orthogonal projection of its width. For such views, the rrvetheds of this article will 
fail. 








still hold in the image formed from V. 

Thw rnn 4 (Skeleton Theorem). Let Tyl u T be the skeleton of E associated 
With lOffle Vantage point V. Then provided that Cp can be thought of as 
being formed by the orthogonal projection of Tyf along the direction to 
the vantage point V r 

(i) Cy is qualitatively symmetric about the image of the a*is A of E, in the 
ion se of theorem 3 

(ii) the image of T consists of one or more connected components,, through 
which A passes, and between any two Df which there exists a mapping that 
is (l - t), continuous and onto, that preserves the gradient of the image of T 
at each point. 

Corollary: If E obeys the prerequisites of theorem 4, and if the image of its skeleton 
decomposes in a unique way into l wo components that satisfy conditions (l} and (ii) of the 
theorem, then these components are the imges of Tp and or T- The avis of symmetry of 
cht image of rp is the image pf the axis of I. 

This theorem makes explicit the many relations between the elements of a 
Skeleton’s image, and its practical importance is illustrated by figure Ii, The theorem states 
that the image of the "sides' -obeys quite welt the symmetry relation of theorem 3, and one 
can see 3 rom the figure that this IS true of the sides of the bucket in the image. The axis of 
symmetry of the sides is the axis of the bucket, The theorem also says that the images of a 
Cones radial extremities are all parallel: to one another, and embrace the COrte's axis. In 
figure n, there is a clear parallel relationship between the image of the bucket’s top. the 
corrugations in its side, and the visible part of its base. 

As in the case of theorem 5, the diagnostic power of this result lies in the earnMary. 
It does not guarantee that a given set of occluding contours can be interpreted, but if a 
unique interpretation exists that satisfies these conditions, then it will be correct. In a real 
image, many parti Of a cone's skeleton will be Obscured, but this hampers the finding of 
relationships (ike parallelism and qualitative symmetry only slightly. One can devise a 
cooperative algorithm fMarr i Foggio 1976) that can Operate on the description of a contour 
to find relationships pf this kind between its pieces (Vatan & Mari 3977), 

Ctf'n^q&ifrf rents uAusr CMKJtttfiim is not comytfx 

We are now- .ready to extend the theory to the case where the cone’s cross-section 
contains concavities. The imporianr difference between this and the case whrre thE crass- 
section is convex is that occluding contours can now also arise from local maxima and 
minima in the cross-section p. For example, in the image of a fluted pillar, there are many 
lines running parallel -,o the axis of the pillar, corresponding to the local maxima or minima 
in the pillar’s cross-section. 

This gives us the extra tool we need to extend the analysis of theorem 4, 
Contours that ate due to convexities and concavities in the cross-Mcrton p behave like the 
fluting on a pllta^ so we tall them the cone's fluting and denote them by the letter t. The 


11- Methods based on the corollary to theorem 4 suffite to solve this senate of a bucket. An 
aiital symmetry ts established by its Sides about Che bucket’s axis (shown thickened), and a 
parallel relations'llip holds between components of Its radial extremity. Hch\ these are Che 
bucket's top and bottom, and the corrugations m its side. 





















fluting on a cone with variable cross-section behaves rather Elks the silhouette of theorem 3 
Convexities and concavities in the Nutmg on one side of the cone’s axis are in qualitative 
symmetry with the fluting on the other side ;ai seen hy the viewer). Thl* meins that 
tan tours in the fluting obey a set of qualiT&uve symmetry and rough parallel relations 
among themselves, similar hut orthogonal to those obeyed by the radial extremities. These 
relationships tan be used to interpret the contours in an image, in a way analogous to 
theorem i. The extension &r theorem 4 to the case of a cone with fluting is theorem 5. 

T defiTfjti J. Let y T li 4> he the skeleton and fluting of 2 associated, with 

some distant vantage point V. Then 
(L) Cff and T obey theorem 4. 

(ii) The image of each portion ffAflJ.pfSjJ, $ L , x), for fixed ^ and varying 
z} of the fluting is either a straight line t ot it divides into convex and 
concave segments that are in {I *■ i) correspondence with the convexities and 
concavities in that part of the Image of fy which lies on the same side of 
its axis of symmetry. 

Coronary: !f T is a generalised cone, and if che contours in its image decompose in a unique 
way into three parts that satisfy the conditions of theorem 5, then those pares are (i) the 
image of Tj/, whose axis of symmetry is the image of the axis of 2'; (ii) the image of T\ the 
rad lsL extremities of 2; and (Hi) the image of <t, the COfle's fluting. 

Once again This result enables Us to set up a system of constraints on the contour 
present in an image such 'hat, if the constraints are satisfied by a particular labelling of the 
contour, chat labelling enables us to discover the axis A of S h and other information about 
its cross-section p and ax ill scaling function A. The algorithms that implement this method 
need only recognise the properties of parallelism and qualitauve symmetry between a small 
number of elements. This result reaches slightly beyond the scope of this article since it 
deals with contour that is not necessarily occluding. Ii also extends naturally to the case 
where p contains creases (points of discontinuity in gradient}, which is helpful because 
creases often give rise to edges and highlights in an image. 

Nearby viewi ng fctiiuf J and cmvtd axi.} 

The methods discussed in this article are ill-suited lo imagrs that arise from 
nearby Viewing points, and are of little use for cones with curved axes unless their cross- 
sectiofis are simple. These points at best made by figure 13, which shows a serpent weaving 
[awards and away from a nearby viewer (figure 13a), who .sees an image that resembles 
figure 13b. The points of inflexion in figure 13b are caused by perspective, and to recognise 
this one needs other cues, like texture gradients and stereopi.it 

if one sees the contour chat appears in figure 13c, one can and does infer The shape 
of h snak€. such ax fugurej He & d, where the scaling function A is roughly -constant., 

are easy to deal with; SO are other cases where the qualitative symmetry Of theorem 3- is 
reversed (ie- convex segments march concave segments, not convex ones), but in general the 
situation can be complex. E have been unable to derive any substantial rtsults from 


12- When [he cross^ecrion of 2 is not convey we need to add the images of Jts maxima and 
minimi in order to arrive at a complete model of its visible contours. The lines traced our 
by these maxi mi and minima are called the surface's fluting, and they are added to the 
skeleton is defined in figures 5 and 10 to produce what is called the surface's complete 
skeleton. This figure depicts a surface with fluting. 























13. The methods af this article are based on the distinction between convex and concave 
contour segments. They are therefore unsuitable for images of nearby objects. For 
example,, if a viewer is dose to a snake (as in 13a}, the image he sees will be somethsn^ like 
]$b. The convexities and concavities in this are mostly due to the perspective 
transformation, and they do not reflect propemes Olf the viewed surface. The figures in 13*t 
and d are generalized cones with curved ajses. It is not known how to deal with these except 
in simple cases tike those depicted here. 


circumstances Lifi which- the surface Jf and its viewing point are unconstrained. 


3< Surfaces eompfl&ed of two or more generalized cones 

We have hitherto been concerned with Che appearance of a jingle generalized 
cone. Reas-llfe object* are often approximately composed of several different cones, joined 
together in various ways (see Marr k Nrshihara L$77 figure 6), and we therefore have to 
study ways of decomposing a multiple cone into its components — for example, a human 
body into arms, leg*, torso and head. The way in which two cones join has a profound but 
usually Setal effect on Lhe contours produced by the resulting surface, and it can upse? the 
qualitative symmetry and parallelism on which our earlier remits depended by interfering 
with the inflexion points on which primary contour description is based. Therefore, the 
algorithms for interpreting occluding contours in an image must incorporate a sensitivity to 
situations that Can arise as a result of joins. 

In this section, we study the common rypes of cone-cone junction, classify the 
appearances to which the)' can give rise, and indicate how algorithms for (heir detection 
may be constructed. In order to do this, we once again have to place some restriction on the 
way in which a join is configured. Thu one that I choose is; 

Restriction 5:Tht axes <?f Cwd Joined genevclited cants art coptanar, 

which enables one to relate the silhouette of the junction between two cones to the angle 
between their axes and their axial scaling functions. If the two axes are not topknar, the 
surfaces at the junction are rather unconstrained- In practise, RS is not a severe restriction. 
Provided that the two ax.es approach one another closely relative to the width of their 
respective cones, the COplanar condition will be satisfied closely enough. 

A: 3idr-tc>-end jo ins fiefm/f’i fm gtJUnhztd cants 

j he most useful common feature of the join between two cones is that it gives rise 
to one or !Wo deep concavities in the surface's silhouette. This feature is unfortunately not a 
necessary concomitant oF a oshf-egne junction, and although it plays a large part in out- 
algorithms for detecting such a junction (Vztan 197S), its role in the underlying theory is 
surprisingly slight. 

It is convenient to divide the types of join that can occur into two classes, those in 
Which the end of One cone is attached to the side of the Other, and those in which the two- 
cones are attached at theur ends. The two types of join are Illustrated trt figure H, and a 
formal statement of Jhe distinction between them is given in the appendix. These two cases 
ate not quite exhaustive, but the intermediate cases introduce no new points of interest. 

From the point of view of diagnosing these joins, the important difference 
between them is that there are often iwo concavijies associated with a side-end join, tone on 
each side of A^ as shown in figure Ila}, but there need not be for end-to-end joins (figure 
idb). We analyze ihe possible configuration* case by case, 


14, The Two main l^pci of joins considered Jfi this article.. E4a ihowj a ilde-Lo-end join, and 
Mb shows an end-to-end join, In the case of a Side-tO-end join beLWCen two convex cones 
<like t4a) 7 theorem -6 guarantees the presence of a minimal concave angle in the bounding 
contour, due to fhe join itself- Provided that the cones are long relative to their width, the 
total concavity Will be near 130°, 












Al: Both, conej art convex 

An Importim due far joins between cones is the existence- of derp concavities In 
the bounding contour, Figure 14a iELustrites this. Provided Thai the end of one cone- joins 
the side of the other wd'l between its ends, one cannot help forming substantial concavities 
in the outline. The precise result that establishes this for convex cones is theorem &, but The 
derails may be confined to the appendix without loss, it basically states that the tota] 
concavity created by a join like that shown in figure Ha is nearly I8Q a > and this gives us a 
method f-OJ detecting 4UCh joins. Since angles of ISO** are preserved by linear 
transformations, the effect of altering the viewing angle is entirely due to changes in the 
angles that are caused by foreshortening fj o: of figure This means that the join 
will remain detectable until the projection of one cone's length becomes comparable with the 
projection of its width- (when (he view becomes "untonventionaD, or until the junction is 
obscured, 

A2; Cones flOf eEWTnrfiFTf con near 

If the generalised cones 2^ and Ej are not -convex “for example, if their axial 
scaling Functions contain concave segments — the concavities that "ought " 1 10 arise at their 
junction can be concealed, appearing as part of the concavities due to their axial scaling 
functions. The simplest case of thjs is shown in figure 15, where there is no identifiable 
concavity due to Che join. Therefore, although concavities provide our algorithms with 
useful first places to look for joins, we need somewhat more solid results- on which to base 
the underlying theory of join deletion. 

The approach we take for diagnosing joins U similar to that of theorems 3 to £ r 
We establish a see of constraints that are satisfied by the different types of join, and argue 
that, if the contours in an image decompose into segments that satisfy these constraints, then 
they may be interpreted as two joined generalised cones. If there is only one decomposition 
of the contours that satisfies these constraints, then the interpretation is also unique. The 
relations involved are usually -quite sLmpje. We shall assume that holds throughout this 
section. 

Suppose that an end of Ej joins the side Of I|. and that the resulting surface is 
viewed along the direction perpendicular to the planes of the axes. If the angle u (figure 
t4)- between their axes is small, or if the line of sight lies tan near the plane of the axes, only 
one "side" -of each cone may be visible (figure IS), In such cases, there are no symmetry 
relations in the image, and the cones' axes cannot he found. 

Provided that both sides of the conei remain visible (figure E7), convex and 
concave segments that lie distal to the join Ire Uninfluenced by it and will obey She 
symmetry theorem 3. In this way, rhe distal segments of the cones determine their axes, 
which can Chen be extended biCx to the join (shown dotted in figure 17). 

This diagnostic technique relies on the existence of segments distal to the junction, 
sa we now deal with the case in which there are none. If we assume that the jiain cakes 
place entirely within one segment of 2^, there are six possible situations and they appear in 
the IOp and bottom rows ef figure 14. Four arise when £j and 2^ consist of just one segment 
each, and it may be either convex nr concave; ihe-Other two arise because can straddle a 
segment boundary in 2^ (column 3 of figure IS}, It is convenient to subdivide the cases 



15. joafiS between cones can easily b t hidden. Here, concavities chat "ougJit H 10 arise as a 
reiylt oF the junction are hidden in concavities that are caused by the sk:s 1 scaling FuncEL&m. 
This is Why iiraigbcforward methcdi for finding joins between cones fail in the general 
Case. 







36. 3f two canes jain at so oblique an angle [hat the structure of one is intermixed, with ihe 
Structure of the other, K (nay be impossible Co recover their axes from the image. This 
figure Shows an exanapte in which iheconej of L6a and 36b are Joined to form. 36c. Methods 
based an the theory given here would produce the deajmpnsition shown in S6d rather than 
that of 16c, because there is no detectable symmetry about the axes depleted In |6c. 























17 If two joined cones are bath long, which means [hat they both contain more than one 
segment, distal so the join, their ajtes ■tan be rcco-vEred by methods based on theorem 3 that 
tate no actOLint of the join. The axis fragments thus obtained, shown SfrlJd, can then be 
extends! to them ijUersectaon point along the dotted lanes, and the jean Itself Can be analyzed 
after this, 



where E^ is concave into those cases tn which the value or fig. passes through a minimum 
and [ficn increases as one moves diisally from Ij (bottom rnw), and [hose in which [he 
minimum value of ^ h achieved at the distal end of Aj (middle row), ]n Figure l£, this. 
minimum value is ieiu. which produces a cusp-like Ej- Nonce chat case ISa exhibits the 
Situation described by theorem &, which guarantees the presence of the concave segments 
joining Ej and f°r reasonable values of fj and &q, 

We are now ready for the main result about iide-ro-cnd joins. Figure IB explains 
what is happening. 3n each of the cases shown there, segmentation points P and Q can be 
found that decompile the contour jo as to satisfy a number of relationships. Theorem 7 
defines the segmentation points precise])' by making these relationships exp Licit. 

t htOrrm ? (Sidi-to-cnd joins). Let Cy be a connected Contour bounding the 
image of two generalised cones Jj and £r, connected by a side-to-end join. 

We assume that the image is formed from a distant viewpoint chosen such 
that the viewing direction lies perpendicular to the plane containing the two 
cones' axes A| and Aj- Assume that Cy is broken into segments at points of 
inflexion. Then there exist two points P and Q each of which u either a 
point of inflexion or lies within a concave contour segment such that 
(i) The line PQ lies within the figure hounded by Cy 
(i0 PQ divid&s Cy into two parts Cj and C^ r between the contour segments 
or fragments in each of which there exists a qualitative symmetry whose 
two axes are the images of A| and Aj 

(iii) P and Q minimise the length of contour fiagment left unmatched by 
these symmetries 

(iv) contour Fragments in Cj left unmatched by fire symmetry round A[ 
would be marched by Contours whose proximal parts, and possibly all of 
Which, lie in the interior of Cj o PQ\ and vict viTsa. 

O) the image of Aj intersects PQ between P and 
Corollary; If the points P and Q are unique, these constraints determine a unique 
decomposition of Cy from which images of the two axes A L may be recovered, 

In practise, it does not matter if P and Q are not unique provided that all possible choices 
give the same axes. 

E. Taiu generalized amis joined end-fCHflrf 

If £] 01 2^2 containJ more than one convex Or concave segment, that Cone’s axis 
may be found for segments distal to the join, just as they were found in figure 37 fnr side- 
tp-rnd joins. Hence we need -consider only the case where £j and £ 2 have jpsf me segmem. 
Once again, the main result depends on chaiacteriiing the segmentation points P and ■£, 
and figure is gives examples Of Segmentation points for end-to-end joins between the 
various types of single segment cone. Theorem & defines these points precisely; it is very 
Similar to theorem 7, 













]S. If Th* joined cones are ihon. (he method of figure IT cannot be used. This figure 
illustrates rhe t^pei of sidiHo-end join that can occur. in the first column, the left-hand 
cane is convey in the centre column it is concave, and in the third column, it is convex on 
one side of the join, and concave on the other. The other cone is convex in the top row, 
and concave in the other two, Segmentation depend* upon finding the points P and g, 
which are defined in the text by theorem T and Illustrated here for each cas*. 
































Theartm & (Znd-t^tnd jatm), Let Cy be a connected contour bounding the 
imago oF two genera Hied cones I] and connected by an end-to-end join. 

We assume that the image is formed from a distant viewpoint chosen inch 
that, the viewing direction lies perpendicular to the plane containing (he 
3xes r ^ c [1VCI “nei A] and A-j>- Assume that Cy it broken into segments 
at poinb of inf lex loti. Then there exist two points P and Q in Cy such 
that: 

(i) Either may be a point of inflexion, one fhut only one) 1 may lie within a 
concave segment. and one (but only one} may tie within a convert segment. 
fiO The line PQ hes wuhtn the Figure boonded by Cy 

(iii) P-£? divides Cy Into two parti -Cj and C%, between the contour Segments 
or fragments in each of which there exists a qualitative symmetry whose 
two axes are the images of A] and Ag 

(iv) P and Q minimise the length of contour fragment left unmatched by 
these symmetries 

(O contour fragments in Cj left unmatched by [he symmetry rO’Uhd A[ 
would be matched by a contour whose proximal part* at leait lie in the 
interior of C% u Fgi and nice v-ersa. 

<vi) the images oF A] and A^ intersect PQ between P and Q 

Cvrotlarj: If the points P and £ are unique, these constraints determine a unique 
decomposition of Cy from which images of the two axes. Aj may be recovered. 

Ax/ffljtirti to mm whtu ism* e-iratrouf srpirntj iimigfit Una 
The assumptions R1 - R4 that were made about Ej and I 2 , excluded cases where 
these surfaces contained straight lines. Such cates are frequent in ml life, however, and 
scune examples are shown in figures I9e and f. E9e is a limit of I9d, and in some sense also oF 
13a; I9f is a limit of all of rhe cases. I9f may be solved in the standard way; Q i; the only 
concave point in the contour, and it matches either the point /\ or it induces two “nearest" 
points P ( and P% that separate the two arms of the figure From the rectangle £JFjFF 2 . BdL ^ 
segmentations are permissible. 

Case 19e is more difficult. The only crue inflexion points are fj] and Qg, but the 
line lies outside the figure. If Qj and are used despite this, the segmentation to 
which they lead corresponds to thinking of the figure as a rectangle with a piece excised (cf 
Hollerbach 1975 p. 55). This would be the preferred description if ^ and £J 2 near 
Since ]$e may be regarded as the limit of 13d, the point P (a corner joining two Stalght tines) 
can be regarded as a segmentation point, like the point P in I9d. P chen induces the point 
q as shown, which segments rhe figure in the -same way as EM, When designing algorithms 
for dealing with cases where some lines are nearly Straight. *convex“ Corners often acquire a 
•dtlal statu* mat arises from regarding the Straight lines as limits of concave rather than 
tor vex contour segmen ts. This means in practise that straight lines are somewhat more 
difficult to deal with than curves since, in the initial state of the algorithms for 
implementing the methods defined here. Straight line* and Che comers ro which they lead 



19. 19a to d exhibits the possible types of end-to-end joins, for cones that contain only one 
segment. The segmentation points P and (2 are defined by theorem 8, and illustrated here 
for the different cases. They provide the basis for interpreting the join from an image. 19e 
& f exhibit the configurations that arise in limiting cases where some or all of the contours 
become straisjht lines. Notice that in I9f, the symmetry relations have degenerated into 




































may be allocated with sever*l possible la helling*. 

i 

C: Joint bttviirn mart than Hut getizraliitd. canfi 
The principal difference between this and case B above ii that a given point of 
segmentation may have more than one match elsewhere, For example* in the silhouette of 
an octopus, the deep concavity between each tentacle matches two others. Also in rhii case, Jt 
is possible to have end-to-end joins in which both P and £J he within concave contour 
segments. The only straightforward remit about the case of multiple joins holds when all 
the joined axes art- cnplanai, which Is a common but restrictive- condition. In this case,, the 
relevant result is sp similar to theorems 7 and 6 that 1 omit it. 


4! Discussion 

The purpose of this article was. to elucidate the assumptions that can reasonably be 
made when interpreting the occluding contours in an image. The assumptions at which we 
arrived were stated as restrictions R1 - R4 t and it was then proved that these restrictions 
have a dose relationship to the assumption that the viewed surface is composed of 
generallied nines, In the second and third parts of the article, we took this result as an 
assumption and studied properties of images of surfaces constructed, in this way. We found 
that many constraints hold among portions of the contours in images of such surfaces, and 
that rough symmetries are formed around the image Of the cones' axes. The importance of 
these relations is fhat One can use them to design algorithms for finding the generalized 
cont-based description of a contour, and for extracting any axes that may be presEnt. Ry 
applying these algorithms repeatedly to the contours found in an image, one can often 
dem-c the 3-D model representation (M>it & Nlshlham 1377} of a surfaces shape without 
prior knowledge of it. Methods based on the present theory will however fail for views in 
Which one or more axes are foreshortened (roughly, whenever the condition i.jija(x) > d of 
page 17 is violated). 

The theory presented in this article Li a pure competence theory, or a theory at thE 
topmost Of Marr St Poggio't (1076) four levels, It is concerned with ends not means. The 
natural division between means and ends Is inierestmgly illustrated by the methods for 
Segmenting a oonwur into rwo component generalized can« (theorems 7 £ g}, The Starting- 
point Tar OUr algorithms that actually find the points P and G as defined by these theorems 
is the examination of deep concavities in the contour Cy (tee Vatan 1076), This contrasts 
strongly with th« theory, because the concavities may be small nr even absent, especially for 

end-to-End joins, Only in certain circumstances sews the underlying theory guarantee their 
presence (theorem &), 
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Appendix 

Suppose that E is an arbitrary three-dimensional surface, and that S v is to image 
from viewpoint V as produced by the projection ty. I ™> Sy (figure 2 in the main text). 
5y has a bounding contour C y say, that corresponds to the silhouette at E P and which we 
can think cf as having been obtained by acting on S y with a boundary operator 3 {see- 
figure 2 ). 

Definition, Let Ty be the set of points on 2 whose image ties on Cy, 

Then Ty is ailed the Crnifour gen erflJjjr of Cy On £. 

That is, Ty is the set of pranks P on £ such that ty(P) tics on Cy. We can now define the 
operator &i h Ej * IV which u induced by S, and which selects Ty out of 2. This is 
illustrated in figure 2 r where d is defined is such a way that the diagram in figure 2 

commutes - ie. iy d - Sty, Notice that ty, Cy, Sy and Ty all depend upon the Vantage 
point V, 

A formal statement of the restrictions R1 * RS is now given: 

Restriction RL I ij everywhere twiee differentiate SJ£fA fJfttinufl^j second derivative. 
Restriction R2;7'he inverse ty ^ Cy jj 

Restriction RS.'Tfit mapping ty ; —> C y is epniinttoux. 

KfJrriCiion R4:The confOtir gmrrcWr Fj/ of Cy is planar. 

Af j|r(e/ton Rf.-fhe iferrs of tm joined gmenthted CDJitfj are co^fltior,. 


Lemma L L«jT* f y) - 0 describe a planar curve that is twice differentiable 


with continuous second derivative. Le: L ,* jf? — > be a nan-angular 

rinear transform of fhe plane. Than L preserves points of inflexion in /. 

PTOB/iYft define g - Lf the tmage of/tinder L, by gix, y) - ffL^x, y)). Sine* L is linear, 
non-singular, and therefore continuous, it induce? * (1 - 1) oorropondenM bcwMi the 
slapw of tangent* to/ and tangent! to g. We «n represent the set cF possible ibpe? b> the 

unJt Circ]e and ss L if,duc « a ■ J 1 —> &K which is (I -1) and onto. Hence L* is 

moficnofipc (either increasing or decreasing) - that is, if ( 3 lies between flj an d $ 2 , and 

i^I ^2l < ^ '^si 1 between and Now a point of inflexion X on/ 

is a stationary point for the slope of the tangent* to/ Because L* is monotonia L(X) is 
therefore a stationary point for the slope of the tangents to Lf - g, Hence tfjfj is an 
inflexion point on £,. 


I have Used a geometric rather than an analytic argument brause it is dear how 
the same argument applies ra the case where / is piecewise linear. In this situation,, the 
analog of an inflexion point is a point where the sign of the change in gradient reverses, 
and the argument used here still applies, 

Definition. Let p{r f fi) - 0 be a simple dosed planar curve that is twice 
continuously differentiable; and let h be a twice continuously differentia hie 
positive real function. Let A he a line at lome angle ^ to the plan? of p, 
and denote positions along A by the variable at. Let 3 be the surface h x p. 

Then 3 is a gtruraliitd cam with axis A, eran-stcut,^ Pj tailing.Junction A 


and tcttwrkuy f If } - T /2, £ will be ailed a rigfir gtwdutd a**. (See 
figure 5 in (be main bent). 

De/iixfffou,, Let ^ be a distant vantage point for the generaliied cone £ 
iueb that (i) the image formed from V is ah orthogonal projection, and fia) 
the rays Jn the projection alE lie parallel to the plane of the erois-sectiwi of 
2, Let the direction of these rays in the plane he denoted by the angle £. 
When these restrictions are in effect, we shin denote the contour generator 

iy 


Theorem t. Let £ be a genera liied cone with convex cross-section r - p\Sl 
Then £ satisfies Rl everywhere, and for alt orthogonal projections parallel 
to the erass-section p, it satisfies the conditions R2 - R4. Conversely, if Rl 
is satisfied by the closed surface £ r and if Rl - R4 art satisfied for all 
orthogonal projections parallel to some plane H, then £ is a generalized 
cone with convex generating crniiechori p that lies parallel to H. 

Proof: Om definition of a gen trained cone ensures that it satisfies Rl Since £ k generated 
by moving p along the axis A (see figure & a given rad.al PC - (p(§\ B) sweeps out a 
plane that contains A. as p itself is moved along A. As p moves, the radial PC maintains its 
direction, but shrinks or expands in a manner dictated by the staling function h(t). As C 
moves, it Traces out a curve on £, which we shall call Tp/, and which lies in the plane A, 


Furthermore, :he tangent to E it G that lies in the plane of p is the tangent GV re p at G r 
for all position* of G. Suppose that we represent the direction of GV in the plan-e of p by 
If one Views E from a great dlaance in the direction the line GV is a line of sight CO the 
edge of the surface E. Therefore G lies on the contour generator for this view of £. But this 
Ls true for all positions of PC as P moves along A, and so Ty is a contour generator. In 
fact it IS r + . Furthermore, since p is eon vest, the same will be true for every angle $ ars d 
corresponding pome Onn^ provided that the viewing dirKtions he parallel to the plane of 
p. Hence E satisfies R2 - R4 for all such orthogonal projections. 

The proof of the convene result it longer, and we first need to establish three 

lemmas. 

Lemma it convex for all planes parallel to the plane H of the 

given viewing directions. 

Proof: Suppose that S fitf were not convex. Then there would exist a line in that was 
tangential re E o at two points C f and G% say, as shown in figure 20, But the line Gj<T> 
ti the ray that produces the edge of the image of E from this viewing angle, and G\G^ 
therefore projects to a point P say, on Cy. &o ty'HP) would contain both G [ and Gg, and 
io would not be single-valued. Thiscontradicts R2, 

Aemitid 3. If fwo distinct contour generator on E intersect at a point X, 
then contour generators fur all distant viewing directions in the plane II 
pass through X. 

Proof: The tangent plane to E at X, which exists by R} t contains two disctncc vectors that lie 



30. Lemma 3 ahQWi 'how restriction R2 f&rcss jj Co be tonves. 



in a plane paralkJ to n. Henee [tie tangent plane at X must itself fee parallel to IL 
Ltmrna 4. Let and be contour generators for two different 
viewing direction! in II. Then T^ t and P^ intersect on I. 

Prcof. Since £ is a dosed surface* the Tq for any angie & divides £ into two component!, 
This follow! from the fact that if £ is the surface defined by the equal, m/*, y* z) - Q t the 
points on P^ are solutions to the equations 

Gjadif} . (V ■ (X, % *}} - Q U ) 

/*■ y**) - o (2) 

where ^ is a distant vantage poLnt along the ray! of the orthogonal projection P^, and the 
two components correspond to points where Equation (Ij cates values * 0 and < 0 respectively. 
Hence and P^ each divide 2 into- two connected components. Let TI^ be any plane 
parallel to H that intersects Z in more than a point, Jntfiii simple closed convex curve, 
which meets in Gj and G s , and which meets an G^ and Cj, as illustrated in figure 
21. The tangents to £ at G] and G^ are parallel, and so are the tangents at and G^ 
Clearly, the .me GjG s divides the simple closed curve E n n ,f into two- parri, in one of which 
lies C 2 , and in the ocher of which lies c + But G 2 and both lie on , where*! G l and 
Cg, both lie on r Hence and 1*^ must intersect somewhere on £. 

CmU»r ? .T 4ll and P^. intersect twice or more. 

We can now complete the proof oF theorem L Let T*^ and P^ be two contour 
generators for £ for different viewing directions lying in H Since and P^ are both 
ptanar (by R4), their containing planes internet in a line A (say). By lemma 4* T+ and 


G> r 



G 3 


2]. Lemma 4 establishes thai centaur generators far two different viewing angles must 
intersect. 







22. Diagrams for the proof of theorem I. T/ and are two planar contour generators 
whose planes intersect In the line NS. The -haded region Is parallel to the plane of the 
viewing directions. Figure 22b shows a view this region from above. 


G 

















I>, intersect in at least two points, and these points must therefore lie along th * line A. Lee 
Af a boundary point of the set of interaction points of and r^, on A, and let S be 
the next closest such point to N. Thu situation u depicted in figure 22- By lemma 2, all 
contour generators pas; through N and S, which we may therefore think of as north and 
south poles of I, and therefore the planes of all cauntotir generators for views of E from H 
mu it contain N and 5 and hence the line A- That is, the planes of all contour generators for 
distant views from the plane H win intersect in A- 

Let be a plane parallel to n lying between N and S, distant z from N. as 
shown in figure 22a, Let intersect A at H, at G fcl and 1^ at C%. The 
configuration in H 1 ' is shown in figure £2b. The crucial step in the proof is to nonet that 
up to scalar magnification, the geometry of figure 22h is independent of z, the position of 
II 1 along the line NS* This follows from the following observations: 
fij The angle between HG j and ffGg is independent of z, because it is simply the angle 
between the planes of and F^ 2 measured parallel to Ifc 

(il) The direction of the tangent to p at G| is independent of a, because as r increases, Gj 
traces out the contour generator F^ Jh which is by definition the locus of tangents to X 
parallel to H for a given fixed viewing direction ^ 

We deduce that for each angle @ in figure 22b. the tangent to the curve p has a 
constant direction For every t That is, for each t, the cross-sections p of E in itf are all 
solutions of the same equation 

r.d$idr - (I) 

where / is some function of the viewing angle £ and j; independent of z. Let ft(i) be a 


solution to (l). Then the cross-section function p of E has the form 

pti *j * H*m»} m 

where h Is a positive real function of %. Finally, E is twice continuously differentiable, by RL 
Hence h is a twice continuously differentiable function of z, and so is R of 9. This completes 
the proof of theorem I. 

Ctrollary i: A " 0 at N and at S. 

CorylteTy 2; The cross-sectinji p can change at the poles where h - G„ 

TAewtm 2. A. necessary and sufficient condition for E to satisfy RI, and 
- R4 for all distant viewing positions, Is that I be a quadratic surface. 

Prwtf. i f £ is a quadratic surface, it satisfies R! - ftf. This follows from the following three 
Observations: 

(i) A sphere trivially satisfies RI - R4 

(ii) Any linear transformation or translation preserves RI ’ R4 

(iil) Any quadratic surface may be derived from a sphere by a linear transformation and a 
translation. 

Conversely, suppose that E satisfies RI - R4 for all Viewing angles. Since She 
conditions of theorem I hold for every viewing angle, E may be thought of as a generalized 
cone with generating crass'srctions in any direction. Hence, any two parallel planes intersect 
£ in curves that, if noc null, have the same shape. Suppose that E is the surface represented 
by the polynomial _/?s, y r t) * 0. Then a set of such parallel planes is given by the family 
e - ♦ by * c, for varying c. We deduce that the curves 


fl Jf r y. ax + * cjj - 0 

/fa, J, ax + ) - 0 

are identical, iJp tb magnification and a translation, Hence the cannot multiply terms of 
second nr higher order. Therefore, in the Original equation, 1 cannot multiply terms of 
second or higher order. S-ince the condition holds for arbitrary planes,, it must also be true 
of x And of y, and it must hold identically. Hence/is at most quadratic, and we have 
already seen that quadratic surfaces satisfy the conditions of the theorem, 

■fiemffrjij about tkt proofs 0 / 1 and 2 

The premises of these theorems, and the way the premises are used in the proofs, 
seem generally reasonable with one possible exception. That is the use of R2 to show that 
X n ft/ is convert in lemma 2. In fact, this is how R2 Forces the cross-section of E to be 
convex, which is a condition that holds all through these remits. One might argue that this 
is a somewhat artificial use of R2 r which was introduced mainly to exclude surfaces chat 
contain lines. The analysis given so far concerns only the silhouette or the image of X 
however, and in these circumstances the Convexity assumption fur p is a reasonable one, 
because violations cannot bf detected from viewing directions that he parallel to the plane 
of p. In practise, one can apply the above analysis to all occluding contours in an image, 
provided that one subsequently relates the different cylinder descriptions that emerge fur 
different parts of the same surface. Figure a shows a simple example of this. 


Th ear cm 5 {Ax-lnl SytmtitiTy), Let 3! « p x a he a generalized cone with 


cdfivex erdsi-Mction p, viewed distantly from lLs awocitced viewing plane 
H. and let rhe cross-section Mating function h(%) contain at least one 
concavity,. Then 

(0 the silhouette of 2 decomposes into n ► 2 contour Moments by splitting il 
ic points of inflexion; 

{iij the image of the axis A of £ establishes an axial symmetry between at 
least 2 or (a - 2) (whichever is greater} contour segments, including ail 
concave segments, such that segment! that correspond under the symnwiry 
are either both convex or both Concave; 

fLJl) if £[j and c^ are corresponding segments, and If d(cj) denotes the 
average perpendicular distance from r ( to the image of the axis, then 
dlc^jldic^y) is independent of L 

Pro&f; Since £ it a generaSiMd, cone and p is convey the contour generators for its image 
are the curves 2 ) and (ATa^d^, $ 2 ■ ^ for f ' XEd ind &2 irid variable f H 

as shown in figure 7, Hence as A(z} increases, the distance from contour generator to the 
image A 11 of the axis A increases on both sides of A*- Hence if A(z) has a concave portion, 
it will generate two concave segments c } and e 3 in C^, one either side of A* ami symmetrical 
about A* (with l possible lateral displacement unless 2 is a right general] iedi cone), Also, 
dfei )id(c%) depends upon the value 0 / and not on the particular choice of 

contour segment (i.e. of e) r 

These remarks hold for all convex and concave segments of A r except possibly for those 
at each end of E (see figure 7}- In figure 7, there is effectively only one end segment. 


nimety c^, but in general there map be 2. Hence the symmetry established by the axis A* 
holds for at kaU (n - 2) of C^'s contour segments, {or 2 of [hem If n - j) h ari d jt matches 
Up u 11 contour segments that are concave, since the end segments. must be carves. Finally,, 
there must be at lent three contour segments, since the premises of the theorem require that 
there be 2 concave ones, and » a simple dosed curve. 

Definition, The type of symmetry established by theorem % which hold* 
between convex or concave segments of a contout and which indudes a 
Kaling factor, we shall call a qwUiative symmetry. 

Definition, The set T of pomti for alt 6 and each value of z t that 

mak.H a local maximum dr minimum. Is Called the radial tZItemity of 
I. Let V be an arbitrary distant vantage point, and let yf be the projection 
from V onto a cross-section plane Of I, as described in the main tent and 
illustrated In figure 9. The set fj/ u T is called the tktkstm of X for the 
vantage points, 

’T^eprfjtt 4 fSAsf-eron. Tkturem). Let Tyf v T tie the skeleton of ^ associated 
with some distant vantage point V, Then provided that Cy is the projection 
of Yyt to V, 

Cy ii qualitatively symmetric about the image of the axis A of E. in the 


sense of theorem 3 


(iO the image of T consists of on* or more connected component*, through 
Which A piiM3 r and between any two of which there exists a mapping that 
i-5 (1 - fX continuous and onto, that preserves the gradient of the image of t 
at each point. 

Prwf:Ty : correpond* to a viewing direction that is ccpkanar with the cross-setnon p. Hence 
the contours Cyl in the image as wen from V' obey the conditions of theorem 3*. and the 
image of the axis A induce* a qualitative symmetry between its concave and convex 
component*. Such relation* are preserved by an orthogonal projection, and since it is a 
condition of the theorem that Cy coincides with the projection of Tyf, it follow* that Cy is 
also qualitatively symmetric about the image of A- Secondly, f trivially consms of one 1 or 
more connected eomp&henti through which A passes, Since these component* are just cross- 
sections at the local maxima and minima of A Finally, if AfajJ and art local maxima 
or minima of A, the mapping between points on E given, by 

8, 2 ,J —> [ A(z 2 ).p(»h t. 1 $ 

is continuous, (I - 1). onto, and preserves the gradient of T at each point, since the gradient 
Of both at 0 IS p4pl48. This Correspondence is preserved by an orthogonal projection, and 
hence the relations will still hold in the image of T. 

DtJhiUin. Let I be a generaliwd cone, and let # be the set of points 
(h(x).p(0^) r 8 ir i) tor all i, for each 8^ that maxes p a Local maximum or 
minimum. Then * is ailed theof £. For a viewpoint V t let Tyt u 
T be the skeleton of E that occurs ih theorem 4. We define the anptttt 


iktliton by adding tht fluting to S'* tkeleton, ri.e. the complete skeleton of I 
from viewpoint V n the set Tp/ u T v 

TAeortm 5, Let 0 T u f be the complete skeleton of I associated with 

some distant vantage point V. Then 

(1) Cy ami the image of T obey theorem 4 

GO The image of ea^b portion [ftfrJ.pffjJ, flp e>, for fisted. ^ and varying 
e] of the fluting of E is either a straight line, or it divides into convex and 
concave segments that art in (J ■ J) correspondence with the convexities and 
concavities m char pan of Cy which lies on the same side of its axis of 
symmetry.. 

Pr&cf; Part (i) follows ftbm theorem 4. Part (ij) follows because like the iwc qualitatively 
symmetric components of Cp, each contour in the fluting of 33 is generated by vitiations in 
Mzl If such a contour lies directly on the line of sight to the axis A of E. it will appear in 
the image as a straight line. Otherwise, its concavities and convexities will follow those of 
□ne component of Cy, although the depth of the concavities nr convexities will differ in 
general, 


OtjlTtfuen. Foe t - j, 2 let Ej be a gen-era tiled cone with maximum width 
2iu| and axis A, of length fj. Let Ej and E*j be joined, and lei u be the 
angle between their axes (see figures H and 25). Then the join between she 
cones will be called tidfjo-md {the side of to the end of provided 


(i) the two as.es intersect between the ends of A[ 

(il) the whole of the Joined end AB of Eg Lies between the two lines 
perpendicular to and passing through the ends of A^ 
fijj) due of the point! A and B of figure [4 does not lie within the convex 
hull of E|. 

The Join is called md-tMnd if 

(l) the two nearest ends of A| and A? are within raiit (atj, u^J of one 
another and 

(id) the two furthest ends are greater than fn^ * apart. 

TActjrem 6, Let Ej_ and Eg be two convex generated cones such that the 
end of Eg Joins the side of Ep and the join satisfies restriction R$, Let thr 
Lengths of the cones' axes A^ b* J ip and let the diameter! of their cross- 
sections be bounded by fd * l r 2). Then 

(I) the only concavities that can occur in the image are due to the junction 

(II) viewed distantly perpendicular to the plane of A] and a^ ? the totali 
concave angle present in the image it near 1B0 Q (in the sense made precise 
in the proof) provided that the site of the join is not near an end of Z], 
and that 2) and Eg are much longer than they are wide. 

Proof.-Uy lemma h. all contours derived separately from and Eg are convex. Hence any 
concavity in the image of their onion must be due to the way they are joined. If E[ and 



23. Diagram for the proof of theorem 6. The idea is to obtain, Sower bound! for the 
concavities in the outline that are due to the join, The total concavity ts fe: * &*}. whsch is 
near 1$Q°. 




















have coplanar axes and arc Viewed perpendicularly to this plane, the resulting configuration 
is as represented, in f igure 23. The contours shown in thick lines there repeat cylinders f, 
long and uij thick which, by the conditions of the theorem, bound the cones X % (i - l t 2). Let 
P be one of the two points at which contours due to S[ and Eg intersect, and let P(j l and 
PQ% be the tangents to £] and Eg at P. Let u be !bc angle between the axes A| and A 2 
E e and Ej, and let and be the angle* that And ^£3 rnal -e with A| and Ag. Then 
the angle between PQ^ and PQg is a ISO 0 - « - ^ ^g) r The corresponding angle a! the 
other intersection pf between contour* of £[ and Ig is tJ - {« ■ is illustrated in 

figure 23, Hence the total concavity due to the Join is a * ** - (ISO 0 ■ fa ■ * ^3 - 

In order to establish a lower bound for the total concavity we need to find upper bound* 
far the angles (i - I to 4). and we can use the convexity oF E^ and to do this. Since 
the scaling Functions of E[ and Eg are convex^ the maximum passible value of is 
(shown In figure 23> h which is exactly - afjxefllWX and approximately 

ton The maximum possible value of ^ is ^j, which is approximately ian~^(w^fd]) 

(see figure 33); and Similarly foT ifand These approximations hold provided that the 
cones are long relative to their widths, and it not near 0 or fj - i.e. the join is not near 
either end Of Ej. 

Theorems 7 and £ were stated precisely in the text and their proofs, which are 


straightforward, are omitted. 


